The present invention relates to a method and system for evaluating the confidence in information received from a user, and more specifically, to using cognitive behavior indicators of the user when providing information as an answer to a question to identify a cognitive trait and to determine a quantified level of confidence for the information based on the cognitive trait.
In active learning question and answer systems, such as those used in and for internet and other database search engines, a semi-supervised machine utilizes an active learning algorithm in which a user is interactively queried to obtain desired outputs as new data points. The information input by a user in response to an automated query is used to improve the relevance and quality of the results generated by the system.
Manually labeling data points in the active learning system can be quite costly and time-consuming. The active learning algorithms generate a query to a user or a teacher to provide such labels. However, the resulting label information provided by the user or a teacher may be uninformative and/or inaccurate due to the influence of external stimuli such as stress, medical factors, ambient noise, other distractions and/or other influences which may affect the accuracy and/or relevance of the user's response at the time the information is provided. The inaccuracy or lack of relevance of the user's response may then in turn be inaccurate and/or irrelevant for subsequent users of the system in the future.